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GUIDELINES:  ASSESSING  USE  OF  INFORMATION  SOURCES  AND  QUALITY  OF 
PERFORMANCE  AT  THE  WORK  SITE 


INTRODUCTION 


The  purpose  of  these  guidelines  is  threefold: 

-  To  demonstrate  how  to  write  task  descriptions  of  vehicle 
maintenance  tasks. 

-  To  demonstrate  a  method  for  estimating  likelihood  of  information 
need  for  vehicle  maintenance. 

-  To  demonstrate  a  method  for  systematic  observation  and 
data-gatherlng  on  personnel  performing  vehicle  maintenance  tasks. 

Methods  for  predicting  and  assessing  technical  information  need  and 
usage  are  presented  here  and  were  developed  by  ASA  in  the  course  of 
fulfilling  the  requirements  of  Contract  No.  DAHC19-77-C-0025  "Predicting 
Effectiveness  of  Manuals  Designed  for  Use  by  Job  Incumbents  At  Their  Work 
Sites." 

These  guidelines  are  organized  in  the  following  manner: 

-  Front-end  analysis  of  vehicle  maintenance  tasks. 

-  Unobtrusive  observation  techniques 

-  Training  procedures 

Front-end  analysis  requires  both  a  step-by-step  description  of  the 
tasks  of  interest  and  a  method  for  estimating  information  needs  to 
successfully  accomplish  the  task.  The  step-by-step  task  descriptions  not 
only  establish  a  structure  for  recording  an  observed  task,  but  also 
provide  the  task  analyst  with  the  specifics  of  the  task  on  which  he  can 
assess  information  need. 

The  method  for  estimating  information  needs  is  the  Information  Demand 
Rating  (IDR)  Instrument.  This  instrument  was  developed  as  a  system  of 
rating  task  difficulty  based  on  the  interaction  of  specific  task  charac¬ 
teristics  and  equipment  design.  The  IDR  instrument  rates  tasks  on  six 
different  scales  which  results  in  an  overall  measure  of  task  difficulty. 

These  front-end  analysis  methods  were  validated  by  ASA  on  a  sample  of 
vehicle  maintenance  tasks.  The  results  of  the  validation  studies  were: 

1.  Reliability  of  task  description: 

a.  Total  number  of  steps  in  task  description  for  10  tasks  by  4 
analysts  -  Chi  square,r15.47 ,  df*27,  p>.95,  no  difference. 
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b.  Number  of  steps  assigned  from  TM  description  for  10  tasks  by  4 
analysts  -  Chi  square-7.19,  df-27,  p>.99,  no  difference. 


c.  Number  of  steps  added  by  the  analyst  (TM  description  felt 
incomplete)  for  10  tasks  by  4  analysts  -  Chi  square-39.55, 
df-2,  p> . 05 . 

2.  Reliability  of  application  of  IDR  for  15  tasks  independently  rated 
by  3  raters: 

a.  Lawless  &  Chatfield  (1974)1  Index  of  Agreement  for 
agreement  within  one  point  between  raters,  p>.95  no  difference 
(Number  of  agreements  within  1  point  -  103, 

Number  of  disagreements  within  1  point  -  60 

JP  of  chance  agreement  3  raters  within  1  point  on  5-point 

scale  -  0.232) 

b.  Lawless  &  Chatfield  (1974)1  Index  of  Agreement  for 
agreement  within  one  point,  test/retest  within  raters,  p>.95 
no  difference  (Number  of  agreements  within  1  point  -  121, 
Number  of  disagreements  within  1  point  -  42, 

P_  of  chance  agreement  for  test-retest  within  1  point  on 
5-point  scale  -  0.52) 

Application  of  the  IDR  system  is  reliable  across  raters  and 
within  raters.  The  raters  were  all  familiar  with  automotive 
repair  techniques  and  had  all  received  the  training  outlined 
on  pp.  31-34  of  this  guide. 

The  unobtrusive  observation  technique  requires  that  the  maintenance 
technician  be  observed  at  the  usual  job  site,  performing  routine  duty 
assignments,  with  minimum  intervention.  That  is,  observers  try  to 
minimize  distraction  of  the  task  performer  and  disruption  of  the  shop 
routine.  This  differs  from  other  techniques  that  require  the  task  per¬ 
former  to  think  out  loud.  Any  questions  the  observer  may  have  concerning 
the  task  performance  are  deferred  until  the  post-task  interview. 

This  is  a  get-down-in-the-dirt  technique.  This  technique  requires 
observers  to  go  into  the  motor  pools  in  work  clothes  and  stand  ready  to 
grab  the  end  of  a  torsion  bar,  or  hold  a  nut  from  turning  as  needed.  This 
approach,  unlike  a  study  conducted  in  the  artificial  confines  of  a 
laboratory  environment,  makes  it  posible  to  collect  data  which  reflects 
the  real  world  state  of  affairs  rather  than  some  theoretical  abstraction 
of  the  real  world.  Complete  instructions  for  observing  vehicle 
maintenance  tasks  and  recording  the  observations  are  detailed  later  in 
these  guidelines. 


^-Lawless,  G.  F.,  &  Chatfield,  D.  Multivariate  approaches  for  the 
Behavioral  Sciences:  A  brief  text.  Lubbock,  TX:  Texas  Tech  Press, 
1974. 
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Reliability  scores  for  this  unobtrusive  observation  technique  were 
obtained  as  follows.  Two  observers  watched  the  same  task  performance  and 
wrote  independent  narrative  observations.  The  observers  independently 
transcribed  the  observation.  Then  the  transcripts  were  compared  for 
comparability  of  breakdown  into  number  of  steps  and  significant  steps 
regarding  behavior  coding.  Agreement  levels  were  greater  than  95%  for  all 
pairs  of  observers. 

The  primary  goal  of  this  training  is  to  reduce  variation  in  the 
performance  of  the  data  collection  from  observer  to  observer.  It  has  been 
demonstrated  that  the  problem  of  observer  uniformity  and  training  is 
usually  handled  best  by  having  the  observer  participate  in  the  development 
of  the  system  of  observation  (Heyns  &  Lippitt,  1954). ^  Therefore,  this 
training  procedure  requires  that  a  new  observer  first  become  completely 
familiar  with  the  methodology  for  front-end  analysis  (including  practice 
writing  step-by-step  task  descriptions  and  rating  tasks  for  information 
demand)  and  with  the  existing  body  of  task  write-ups,  demand  ratings  and 
observations  previously  collected. 

The  methods  presented  in  these  guidelines  are  not  theoretical.  They 
have  been  validated  and  used  to  observe  more  than  300  U.S.  Army  mechanics 
over  a  two  year  period.  Using  these  methods  ASA  has  been  able  to  develop 
a  picture  of  technical  information  need  and  use  for  U.S.  Army  mechanics 
performing  their  usual  duties  at  their  normal  work  sites.  These  methods 
thus  provide  an  opportunity  to  provide  feedback  to  training  and  technical 
literature  developers  on  the  way  their  training  and  TMs  are  actually 
employed  under  routine  job  conditions. 


CONCEPTUAL  FRAMEWORK 


In  order  to  make  a  front-end  analysis  for  any  type  of  task,  it  is 
necessary  to  have  a  conceptual  picture  of  the  basic  nature  of  that  task. 
Vehicle  maintenance  tasks  can  be  conceptualized  with  two  basic  models. 

The  first  is  a  linear  model  of  the  task,  in  which  the  task  follows  (stage 
by  stage)  from  Initiation  to  end.  This  model  is  schematized  in  Figure  1. 

The  Linear  Process  Model  begins  with  the  detection  of  a  problem  with 
the  vehicle.  This  is  the  initiating  cue  which  sets  the  task  process  into 
motion.  Stage  1  of  the  task  is  the  "identification  of  the  goal  action 
required."  This  is  the  'identify  the  most  likely  solution’  stage.  Based 
on  the  data  from  the  Phase  I  research,  Stage  1  is  usually  accomplished  by 


^Heyns,  R.,  &  Lippitt,  R.  Systematic  observational  techniques.  In  G. 
Lindzey  (Ed.),  Handbook  of  Social  Psychology,  Vol.  1  Cambridge:  Addison- 
Wesley,  1954. 
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LINEAR  PROCESS  MODEL 
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INITIATING  CUE 
(Operator  or  Inspector) 


STAGE  1 
(Motor  Sergeant) 


STAGE  2 

(Mechanic  Orientation) 


STAGE  3 
(Mechanic) 


STAGE  4 
(Mechanic) 


STAGE  5 
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Figure  1.  Illustration  of  the  Process  Model  Used  as  the 

Conceptual  Framework  for  Development  of  Front-end 
Analysis 


the  motor  sergant  or  a  technical  inspector.  This  person's  proposed 
solution  becomes  the  mechanic's  assignment. 

Stage  2  of  the  task  model  is  the  physical  identification  of  the 
target  component  or  components  upon  which  the  goal  action  is  to  be 
performed.  Our  observations  indicate  that  the  mechanic  usually  enters 
the  task  process  here,  rather  than  at  Stage  1.  The  mechanic  normally 
receives  his  assignment  as  "replace  such  and  such,"  or  "adjust  such  and 
such."  This  stage  is  the  orienting  stage — where  is  it,  what  is  it,  how  do 
1  get  to  it,  what  tools  or  equipment  do  I  need? 

Stage  3  is  the  enabling  process  which  precedes  the  "goal  action." 

This  stage  involves  the  disconnecting,  disassembling,  etc.  required  prior 
to  performance  of  the  goal  action. 

Stage  4  is  performance  of  the  goal  action  named  in  Stage  1.  This 
stage  is  narrowly  defined,  focusing  on  the  activities  directly  performed 
on  the  component  identified  in  the  task  statement.  It  also  includes  any 
actions  with  test  equipment,  special  connections  or  disconnections,  etc. 
which  occur  during  the  goal  action  process. 

All  activities  then,  which  occur  after  the  goal  action  has  been 
initiated  and  before  it  has  been  completed,  are  considered  to  have 
occurred  in  Stage  4.  In  this  model,  simple  Remove/Install  tasks  may  not 
have  a  Stage  4.  For  instance,  in  the  task  "Remove/ Ins tall  Radiator,"  the 
actual  physical  lifting  out  of  the  radiator  is  simply  the  last  part  of 
Stage  3,  and  the  physical  setting-in-place  of  the  radiator  is  the  first 
step  of  Stage  5.  Thus,  there  is  no  Stage  4  in  this  simple  task. 

Stage  5  Is  the  "button  up"  stage.  Reconnect,  reassemble,  adjust, 
test,  and  any  other  activity  that  would  normally  be  performed  in  putting 
the  equipment  back  together  after  completing  the  goal  action  occur  in  this 
stage. 


Stage  6  is  the  Check/Verify  operational  condition  stage.  This  is  the 
final  check,  after  all  activities  have  been  completed,  to  certify  that  the 
equipment  should  be  returned  to  service. 

The  other  model  is  an  an  operational  task  model.  It  is  a  closer 
approximation  to  the  way  maintenance  tasks  are  performed  in  the  real 
world.  The  Linear  Process  Model  neglects  the  necessarily  recursive  or 
repetitive  nature  of  tasks  as  they  actually  occur.  Orienting  activities 
may  occur  —  then  enabling  activities  —  then  more  orienting  activities, 
etc.  Also,  Stage  1  is  seldom  observed  in  the  motor  pool  environment. 
Instead,  the  result  of  Stage  1,  the  task  assignment,  is  the  starting  place 
for  our  observations  and  for  the  descriptions  of  tasks  as  they  are 
presented  in  the  TMs.  Furthermore,  Stage  6  of  the  theoretical  process 
model  is  seldom  observed  in  organizational  motor  pools  if  it  occurs  at 
all. 
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The  operational  task  model,  shown  in  Figure  2,  resolves  these 
problems  by  simply  classifying  the  activities  involved  in  performing  an 
assigned  task  into  four  categories.  The  first  category  is  initial  or 
orienting  activities.  Orienting  activities  are  the  activities  required  to 
begin  a  task,  but  these  activities  do  not  involve  operations  on  the 
equipment  Itself.  Such  activities  are:  locating/ identifying  the  specific 
area,  component,  or  parts  on  the  vehicle  to  be  manipulated;  identifying 
and  obtaining  special  tools,  test  equipment,  parts,  fluids,  etc. 

The  second  category  is  that  of  accessory  activities.  This  category 
Includes  all  activities  which  involve  operations  on,  or  manipulations  of, 
the  equipment,  but  are  not  the  assigned  task.  Thus,  any  enabling  or 
buttoning-up  activities,  installing  or  removing  test  equipment,  warming 
the  engine,  jacking  up  the  vehicle,  etc.  are  activities  in  the  accessory 
category. 

Central  task  activities  are  the  third  category  and  simply  comprise 
the  activities  identified  in  the  task  title.  This  category  includes  all 
manipulations,  adjustments,  and  alignments  necessary  for  proper  completion 
of  the  assigned  task,  but  only  those  activities  directly  related  to  the 
task  title  component. 

Checkout/ verification  activities  are  the  fourth  category.  These 
activities  are  all  activities  which  certify  proper  completion  of  opera¬ 
tions  or  conditions.  Final  operational  checks  which  certify  that  the 
vehicle  is  again  serviceable  are  also  included  in  this  category. 

After  developing  a  model  of  maintenance  tasks  to  aid  in  writing  task 
descriptions  and  assessing  information  needs,  it  became  clear  that  a  model 
of  the  mechanic,  who  performs  the  task,  was  missing. 

Description  of  a  Model  of  the  Mechanic 

The  last  conceptual  requirement,  before  the  task  description  can  be 
written,  is  a  model  of  the  task  performer.  This  model  of  a  performer  is 
necessary  in  order  to  decide  the  level  of  detail  at  which  the  task 
descriptions  should  be  written. 

Our  model  mechanic  is  a  naive  mechanic  who  possesses  the  basic  knowl¬ 
edge  to  use  the  tools  in  the  mechanic's  issue  and  possesses  basic  skills, 
such  as  bolt  tightening,  spark  plug  removal,  etc.  The  model  mechanic  is 
completely  naive  regarding  Army  equipment,  however. 

Thus,  our  task  descriptions  do  not  instruct  basic  mechanics'  opera¬ 
tions  common  to  all  types  and  classes  of  vehicles,  but  do  instruct, 
locate,  and  identify  steps  for  components  or  areas  of  the  vehicle  to  be 
worked  on. 


6 


OPERATIONAL  TASK  MODEL 


CATEGORY  1 
(Orienting  Activities) 


CATEGORY  2 
(Accessory  Activities) 


CATEGORY  3 
(Central  Task  Activities) 


CATEGORY  4 
(Checkout/Verification 
Activities) 


leure  ?  .(.lustration  of  the  Operational  Task  Model 


FRONT-END  ANALYSIS 


The  front-end  analysis  organizes  and  structures  the  information  about 
a  particular  task.  Thus,  front-end  analysis  has  intrinsic  value  in  pro¬ 
viding  a  standard  of  comparison  for  observed  task  behavior  and  in  high¬ 
lighting  those  places  in  a  task  where  the  task  performer  could  be  expected 
to  have  problems.  The  front-end  analysis  also  is  a  useful  training  tool 
for  observers,  familiarizing  them  with  the  tasks  to  be  observed  and  pro¬ 
viding  them  with  a  structure  for  the  narrative  record  of  observations. 

This  system  of  front-end  analysis  is  designed  to  be  used  by  people 
who  are  very  familiar  with  automotive  maintenance  tasks.  The  person  mak¬ 
ing  the  front-end  analysis  should  be  familiar  with  the  equipment  being 
analyzed  and  should  make  the  analysis  while  in  the  presence  of  the 
equipment. 

The  operational  task  model  and  the  model  of  the  mechanic  are  used  as 
guides  to  construct  step-by-step  task  descriptions.  The  descriptions  are 
all  written  with  the  final  goals  of  the  description  -  assessing  probabili¬ 
ties  of  information  seeking  behaviors  (IS")  and  identifying  probable 
trouble  spots  occurring  during  the  task  -  kept  in  mind. 

In  order  to  estimate  the  likelihood  of  ISB  or  making  an  error  for  any 
given  task,  the  Information  Demand  Rating  (IDR)  instrument  is  used.  It  is 
based  on  the  assumption  that  information  need  is  greatly  influenced  by  the 
task/equipment  stimulus  characteristics.  That  Is,  the  need  for  outside 
information  is  partly  determined  by  the  nature  of  the  task  (e.g.,  is  It  a 
matter  of  screwing  on  a  cover  with  4  screws,  or  Is  it  a  matter  of  making  6 
electrical  and  10  hydraulic  corrections,  and  carefully  aligning  the  cover 
to  mount  with  17  bolts?). 

Information  need  is  also  influenced  by  the  design  of  the  equipment 
for  any  given  task  (e.g.,  is  there  no  possible  way  to  do  it  wrong,  or  are 
careful  alignments  necessary,  and  is  it  necessary  not  to  mix  up  the  4 
bolts  in  the  cover  because  they  are  slightly  different  lengths?). 

On  the  basis  of  the  considerations,  the  first  version  of  the  IDR 
instrument  was  constructed.  Several  revisions  reduced  the  instrument  to 
six  items  that  are  best  able  to  discriminate  likelihood  of  ISB  for  indi¬ 
vidual  tasks. 


Step-by-Step  Task  Description 

The  description  of  tasks  as  found  in  the  TM  are  used  as  a  starting 
point.  However,  those  descriptions  are  more  "memory  joggers"  than  fully 
proceduralized  job  performance  aids  (JPAs).  For  example,  TM  task  descrip¬ 
tions  often  do  not  include  orienting  activities.  The  TM  also  leaves  out 
many  steps  of  "good  mechanics'  practice"  since  these  practices  are  simply 
assumed. 
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Therefore,  the  task  description  found  in  the  TMs  must  be  modified  as 
follows: 

-  Each  step  of  the  task  description  will  involve  only  one  task 
action. 

-  The  addition  of  steps  involving  good  mechanical  practice  based  on  a 
logical  understanding  of  the  task  and  equipment  configurations 
(e.g.,  draining  the  oil  from  engine  before  removing  oil  filter, 
placing  vehicle  on  jack  stands  before  climbing  under  it  to  remove 
the  muffler). 

-  The  addition  of  orienting  steps  ("locate”  steps)  for  each  part  or 
component  to  be  worked  on  if  they  have  not  been  named  in  a  previous 
step.  Parts  or  components  which  are  in  the  same  visual  area  or 
which  serve  similar  functions  can  be  named  in  one  step  (e.g.,  lo¬ 
cate  generator,  air  ducts,  flexible  boots,  and  flexible  connector). 

The  task  analyst  needs  to  keep  in  mind  his  primary  goal  -  task 
descriptions  which  are  detailed,  and  which  are  similar  in  structure  for 
all  tasks  and  all  equipment  -  when  applying  the  above  rules.  An  example 
of  a  task  description  is  presented  in  Figure  3. 


Information  Demand  Rating 


In  order  to  classify  tasks  as  to  likelihood  of  ISB  during  task 
performance,  an  instrument  to  measure  the  information  requirements,  or 
information  demands,  of  the  task  is  required.  This  instrument,  the 
Information  Demand  Rating  (IDR),  is  shown  in  Figure  4. 

The  first  two  scales  of  this  rating  instrument  rate  the  task  on  a 
perceptual  dimension.  They  concern  the  accessibility  of  the  part  or 
component  to  be  worked  on.  One,  can  the  mechanic  easily  see  the  part  he 
is  working  on,  (visual  accessibility)  or  is  it  hidden  from  view?  Two,  can 
the  part  or  component  to  be  worked  on  be  gotten  to  easily,  (manual 
accessbility)  or  is  access  restricted? 

The  next  two  scales  rate  the  task  on  a  cognitive  dimension.  That  is, 
how  well  does  the  design  of  the  equipment  insure  that  the  mechanic  can 
determine  what  the  necessary  task  steps  are  (Clarity  of  Necessary  Task 
Steps)  -  "What  do  I  want  to  do?”,  and  what  are  the  techniques  to 
accomplish  this  task,  (Clarity  of  Techniques  to  Perform  Tasks  Steps)  - 
"How  do  I  do  it?". 

These  first  four  items  are  rated  on  five-point  scales  (easy/hard  on 
the  perceptual  dimension,  clear/unclear  on  the  cognitive  dimension)  for 
the  task  as  a  whole.  For  examples  of  anchor  points  for  these  measures  see 
Figure  4. 
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TM9-2350-21 5-20  Fig.  2-143 
ADJUST  STEERING  CONTROL  LINKAGE 
M60/A1  (ORG) 

Park  and  chock  vehicle 

Center  steering  control 

Remove  transmission  shroud  (Q.V.) 

Locate  transmission  control  valve  body 

Locate  steering  linkage  rods,  jam  nuts,  and  pivot  points 

Check  steering  position  indicator  for  centering  on  stamped  dot 

If  it  is  off  center,  proceed  by  disconnecting  rod  end  from  clevis 
by  unscrewing  bolt.  If  it  aligns,  no  adjustment  is  necessary 

Physically  move  Indicator  to  center  mark 

Locate  first  pivot  point  from  steering  control 

Insert  locating  pin  through  alignment  holes  in  bracket  and  clevis 

If  pin  can  be  Inserted,  go  to  Step  17.  If  pin  cannot  be  inserted 
with  steering  control  in  center  position,  loosen  jam  nut  on  rod  end 
and  go  to  Step  12. 

Remove  screw  securing  rod  end  to  clevis 

Screw  or  unscrew  rod  end  on  rod  until  locating  pin  can  be  inserted 
through  clevis  and  bracket  and  steering  bar  remains  centered 

Insure  that  screw  part  of  rod  extends  past  gauge  hole  in  rod  end 

Install  screw  securing  rod  end  in  clevis 

Tighten  jam  nut 

Remove  locating  pin 

Locate  fourth  pivot  point  (in  engine  compartment) 

Insert  locating  pin  through  alignment  holes  in  bracket  and  clevis 

If  pin  can  be  inserted,  go  to  Step  22.  If  pin  cannot  be  inserted,  j 
to  pivot  point  two  in  crew  compartment  and  go  to  Step  21 

Repeat  Steps  10-16  for  this  pivot  point 

Locate  pivot  point  three  (in  engine  compartment) 


Figure  3.  Example  of  Task  Analysis 


TM9-2350-21 5-20  Fig.  2-143 
ADJUST  STEERING  CONTROL  LINKAGE 
M60/A1  (ORG) 

(continued) 


23.  Pivot  point  three  can  be  reached  through  shield  on  A1  models;  on 
M60  models — remove  power  pack  (Q.V.) 

24.  Remove  cover  from  control  box 

25.  Insert  locating  pin 

26.  If  pin  cannot  be  inserted  at  point  three,  repeat  steps  10-16  for 
this  pivot  point.  If  pin  can  be  inserted,  go  to  Step  27 

27.  Locate  pivot  point  four  (in  engine  compartment) 

28.  Insert  locating  pin 

29.  If  pin  cannot  be  inserted  at  point  four,  repeat  Steps  10-16  for  this 
pivot  point.  If  pin  can  be  inserted,  go  to  Step  30 

30.  Connect  rod  end  to  clevis  by  installing  and  tightening  bolt  at 
transmission  control  valve  body 

31.  If  steering  position  indicator  does  not  point  to  center  mark,  repeat 
Steps  12-16  for  the  rod  end  at  transmission  control  valve.  If  it 
does  point  to  the  center  mark,  go  to  Step  32 

32.  Remove  all  locating  pins  from  all  pivot  points 

33.  Install  transmission  shrouds  (Q.V. ) 

34.  Road  test 


Figure  3  (continued) 
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INFORMATION  DEMAND  RATING 


Perceptual  Dimension 

1.  Visual  Accessibility 


(Can  the  part  or  component  to  be  worked  on  be  seen  easily  -  such 
as  the  upper  radiator  hose?  Or  is  it  hard  to  see  -  such  as  the 
parking  brake  drum?) 


2.  Manual  Accessibility 


(Can  the  part  or  component  to  be  worked  on  be  gotten  to  easily  - 
such  as  the  radiator  cap?  Or  is  it  hard  to  get  to  -  such  as  the 
double  spray  linkage?) 


Cognitive  Dimension 

1.  Clarity  of  Necessary 
Task  Steps 


5 

□ 

unclear 


(Is  it  clear  that  something  has  to  be  done  first  -  such  as  removing 
the  spark  plug  wire  before  removing  spark  plug?  Or  is  it  unclear 
whether  something  has  to  be  done  before  starting  on  the  central 
task  -  such  as  disconnecting  the  battery  ground  before  removing  an 
electric  fuel  pump?) 


Clarity  of  Techniques  to 
Perform  Task  Steps 


5 

□ 

unclear 


(Is  it  clear  as  to  just  what  needs  to  be  done  first  -  such  as  remov¬ 
ing  the  distributor  cap  and  rotor  before  cleaning  the  breaker 
points?  Or  is  it  unclear,  such  as  which  tubes  and  hoses  must  be 
removed  before  removing  the  cylinder  head?) 

Performance  Requirements 


1.  Special  Tools  Required  i  j  [~J 

no  yes 

(Are  any  tools  required  that  are  not  found  in  the  mechanic's  common 
tool  box  -  such  as  torque  wrenches,  timing  light,  feeler  guages?) 


2.  Formal  Specification  Required 


(Does  the  task  require  any  close  tolerances  or  fine  adjustments 
such  as  torque  cylinder  head  bolts,  or  adjusting  valve  tappet 
c  learance?  ) 


Figure  4.  Illustration  of  the  Information  Demand  Rating 
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The  last  two  Items  of  the  1DR  concern  the  precision  of  actions 
required  by  the  mechanic,  (Formal  Specification  Required)  and  any  special 
equipment  required  to  accomplish  these  actions  (Special  Tools  Required). 
Special  tools  refers  to  any  tools  not  found  in  the  mechanic's  common  tool 
box  (e.g.,  torque  wrenches,  timing  light,  fuel  gauges,  etc.)  Formal 
Specification  refers  to  any  close  tolerances  or  fine  adjustments  required 
by  the  task  (e.g.,  torque  cylinder  head  bolts  to  65  lbs.  ft.,  adjusting 
valve  tappet  clearances,  etc.).  These  two  items  are  simply  rated  yes  or 
no. 

The  IDR  rating  for  a  task  is  the  simple  sum  obtained  from  the  six 
items.  However,  an  assigned  task  will  often  consist  of  several  idealized 
tasks  nested  within  it  (e.g.,  after  installing  a  distributor  the  mechanic 
adjusts  the  contact  breaker  points  and  times  ignition).  This  problem  is 
solved  by  summing  the  IDR  ratings  obtained  for  the  three  tasks  and  using 
this  simple  sum  as  the  measure  of  information  need  for  the  observed  task. 


UNOBTRUSIVE  OBSERVATION  TECHNIQUES 

1 

>- 

The  unobtrusive  observation  (minimum  interaction)  technique  presented 
here  involves  a  full  narrative  form  of  data  collection.  A  major  advantage 
%•  of  the  narrative  data  collection  form  of  the  unobtrusive  observation 

technique  is  that  it  provides  the  possibility  of  a  “line  audit”  from  the 

R  actual  observed  task  performance  to  the  step-by-step  task  description 

produced  during  the  front-end  analysis. 


The  typical  problems  with  this  kind  of  approach  (unsystematic  collec- 
!■  tion  of  data,  lack  of  formal  data  structure)  are  overcome  by  comparison  of 

-  observed  tasks  with  front-end  analysis  of  the  tasks  and  by  special  train- 

■ing  of  the  data  collection  personnel.  The  step-by-step  task  descriptions 
from  the  front-end  analysis  provides  the  necessary  structure  for  develop- 
ment  of  the  narrative.  The  system  for  the  observation  methodology  is 
'?■  described  in  the  "Instructions  for  Observers"  (pp.  14-30)  and  the  system 

for  training  observers  is  detailed  in  the  "Observer  Training  Procedures" 
r  (pp.  31-34)  which  follows. 

£ 
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INSTRUCTIONS  FOR  OBSERVERS 


These  Instructions  serve  as  a  guide  for  systematic  observation  and 
data-gathering  on  personnel  performing  maintenance  tasks.  The  instruc¬ 
tions  attempt  to  establish  decision  rules  enabling  the  gathering  of  usable 
data. 


Procedure  for  Briefing  Task  Assigners 

1.  Introduce  yourself.  Tell  the  task  assigner  who  you  represent. 

2.  Tell  the  task  assigner  the  name(s)  of  the  higher  level  authorizations 
and  the  division-level  contact. 

3.  Explain  the  following  points  of  the  study: 

a.  The  study  is  based  on  unobtrusive  observation  of  regular,  ongoing 
work  in  the  shop. 

b.  The  purpose  is  to  provide  information  about  what  the  situation  in 
the  shops  really  is,  as  opposed  to  what  the  situation  in  the  shop 
is  supposed  to  be.  We  want  to  find  out  what  the  problems  are  and 
what  causes  them. 

c.  The  goal  is  to  supply  information  to  the  Army  so  the  Army  can 
Improve  the  way  it  supplies  tools,  equipment,  and  information 
required  to  perform  the  job  effectively  and  efficiently. 

d.  Explain  that  the  goal  Is  not  to  evaluate  the  performance  of  the 
shop,  unit,  or  any  Individual  member  of  the  unit. 

e.  Explain  that  we  are  not  looking  for  specific  or  detailed  changes, 
but  are  observing  tasks  as  they  are  performed  in  the  usual 
manner. 

Brief  the  task  assigner  on  the  kinds  of  tasks  needed  for  the 
observation. 

Explain  to  the  task  assigner  the  importance  of  his/her  cooperation 
in  the  following  areas: 

a.  Assigning  the  tasks  in  the  normal  manner  so  as  to  minimize  the 
disruption  caused  by  having  observers  present. 

b.  Help  in  keeping  casual  observers  away  from  the  task  under 
observation  in  order  to  minimize  disturbing  the  routine. 


c.  Whenever  possible,  assign  a  single  individual  to  a  task. 

d.  Explain  how  more  than  one  person  assigned  to  a  task  or  the 
presence  of  a  casual  observer  can  interfere  with  the  data 
collection  process. 

6.  Explain  the  complete  data  collection  structure  to  the  task  assigner. 

7.  Review  scheduled  maintenance  for  the  week  with  the  task  assigner  and 
establish  a  workable  schedule  for  the  data  collection. 

8.  Emphasize  again  the  aim  of  the  study  to  remain  as  unobtrusive  as 
possible  and  our  need  for  the  help  of  the  task  assigner  in  achieving 
this  aim. 


Procedure  for  Briefing  Task  Assignees 

1.  Be  sure  that  you  are  present,  if  possible,  when  the  maintenance 
technician  is  being  assigned  to  the  task  to  be  observed.  The  task 
will  be  considered  initiated  once  the  maintenance  technician  is 
informed  of  his  assignment  to  the  task. 

2.  Determine  if  any  information-seeking  behavior  occurs  between  the  task 
assignee  and  task  assigner  at  this  time. 

3.  Record  the  information-seeking  behaviors,  if  any  occurs,  on  the 
Observation  Data  Sheet  (ODS)  according  to  the  guidelines  set  forth  in 
the  observation  procedure  section. 

4.  Upon  arrival  at  the  task  site,  find  out  the  maintenance  technician's 
MOS/duty  position  and  record  in  the  appropriate  space  on  the  ODS. 

5.  Record  the  date,  task  name,  time  and  your  initials  in  the  appropriate 
spaces  in  the  ODS. 

6.  Assign  the  observation  number  to  the  maintenance  technician  at  this 
time.  The  observation  number  and  the  observer's  initials  serve  to 
identify  each  task  observation. 

7.  Record  the  assigned  observation  number  on  the  consent  form  and  the 
ODS. 

8.  Explain  to  the  maintenance  technician  how  the  observation  procedure 
is  structured,  i.e.,  pre-task  interview,  task  observation. 

9.  Emphasize  to  the  maintenance  technician  that  he  is  not  being 
evaluated  in  any  way.  Answer  any  questions  the  maintenance 
technician  may  have  concerning  the  observation  procedure. 

10.  Ask  the  maintenance  technician  to  read  and  sign  the  consent  form. 
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11.  If  the  subject  requests  a  copy  of  the  consent  form,  comply.  Be  sure 
to  retain  the  original  signed  consent  form. 

12.  Prior  to  the  beginning  of  the  actual  task  performance,  find  out  how 
many  times  the  maintenance  technician  performed  this  task  prior  to 
the  present  time,  and  how  long  it  has  been  since  he  last  performed 
the  task. 

13.  Record  this  information  in  the  appropriate  spaces  on  the  ODS. 

14.  Ask  the  maintenance  technician  the  exact  name  of  the  task. 

15.  Ask  the  maintenance  technician  to  inform  you  when  he  has  completed 
the  task. 


Observation  Procedure 


1.  Record  all  task-relevant  activities  (as  defined)  in  narrative  style 
on  the  observation  data  sheet  (i.e.,  describe  what  is  happening).  Be 
sure  to  include  enough  of  the  context  that  actions  can  be  understood 
when  the  narrative  is  read  next  year. 

2.  When  information  seeking  behavior,  errors,  and  other  behaviors  occur, 
they  should  be  recorded  in  detail  in  the  narrative  text.  The 
appropriate  code  for  that  behavior  should  be  placed  in  the  "code" 
column  (see  Definitions  and  Criteria  section). 

3.  The  narrative  text  should  include  as  much  detail  about  information 
seeking  behavior  and  errors  made  as  is  necessary  to  actually  describe 
the  event. 

4.  For  information  seeking  behavior,  it  should  include  exactly  what 
information  was  sought.  It  should  also  include  why  the  Information 
was  sought  and  from  where  or  whom  it  was  sought.  Whether  or  not  the 
desired  information  was  obtained  should  also  be  recorded. 

5.  Situations  in  which  information  is  sought  but  not  obtained  should  be 
coded  as  information-seeking  events.  Make  a  note  that  the  informa¬ 
tion  was  not  obtained  by  putting  an  (x)  in  the  code  column.  Be  sure 
to  find  out  and  note  in  the  narrative  what  kind  of  information  was 
being  sought.  Distinguish  information  that  was  not  obtained  from 
information  that  was  found  but  was  not  understandable  or  useful.  If 
information  is  obtained  that  is  not  useful  find  out  why,  i.e.,  wrong 
question,  wrong  model,  etc. 

6.  When  the  mechanic  is  looking  in  a  manual,  it  will  usually  be 
necessary  to  ask  (casually),  "What  were  you  looking  for?"  and  "Did 
you  find  what  you  were  looking  for?” 
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7.  When  recording  an  information-seeking  behavior  (ISB)  involving  a 
technical  manual  (TM),  be  sure  to  note  the  TM  number,  page  number, 
figure  number,  and  TM  date. 

8.  There  may  be  occasions  when  the  mechanic  seeks  information  for  a 
short  time,  works  briefly,  and  goes  back  for  more  information.  If  he 
is  working  for  less  than  a  minute  between  information-seeking 
behaviors,  the  information  seeking  will  be  scored  as  a  single  event. 

9.  Information  codes  are  to  be  used  in  the  following  manner.  Record 
information-seeking  events  in  four  categories:  Volunteered  or 
Requested  /  Internal  or  External  /  Source  /  Type.  Every  information- 
seeking  event  will  include  all  four  categories.  Record  the 
abbreviations  for  each  section  in  the  order  stated  (l.e.,  R/E /$'$:?&), 
See  Codes  for  Narrative  Data  section  and  the  example  ODS. 

10.  When  the  observer  realizes  that  an  error  has  been  made  he  should 
record  its  occurrence  in  detail  in  the  narrative  text. 

11.  The  exact  nature  of  the  error,  the  step  on  which  it  occurred  (if 
possible),  and  whether  the  mechanic  knew  an  error  has  been  made, 
should  be  recorded.  Whether  or  not  the  error  was  corrected  should 
also  be  indicated.  Also,  try  to  determine  and  note  the  source  of  the 
error — the  reason  the  error  was  made.  (Enough  Information  should  be 
included  to  allow  the  correct  coding  of  errors  during  the  Data 
Transcription  Procedure.)  Indicate  error  events  on  the  ODS  in  some 
unobtrusive  way  in  case  mechanics  wants  to  read  the  narrative  text. 

12.  Some  steps  will  require  non-information  seeking  behavior  codes  (see 
Definitions  and  Criteria  section.  Codes  for  Narrative  Data  section, 
and  the  example  ODS). 

13.  The  non-information  seeking  behavior  codes  will  be  placed  in 
parentheses,  i.e.,  (Jl)  (X)  (3)  in  the  code  column. 

14.  Any  behaviors  that  seem  unclear  should  be  flagged  (?)  in  the  code 
column.  This  will  enable  you  to  remember  certain  points  that  can  be 
discussed  with  the  mechanic  during  the  post-task  interview.  Even 
though  we  want  to  be  as  unobtrusive  as  possible,  do  not  hesitate  to 
Interact  with  the  technician  to  have  questions  resolved.  Chances  are 
any  scoring  problems  will  be  cleared  up. 

15.  It  is  necessary  to  the  observation  technique  to  observe  no  more  than 
two  persons  per  task  unless  a  particular  part  of  the  task  requires 
more  than  two  persons.  Do  not  disrupt  the  usual  operation  and  task 
assignment  procedure  of  the  motor  pool,  but  make  the  needs  of  the 
research  clear  to  the  task  assigner. 

16.  If  the  task  assigner  assigns  more  than  two  persons  to  the  task,  con¬ 
sider  employing  instruction  for  observing  teams  (see  Instructions  for 
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Observing  Teams,  pp.  29-30).  If  team  procedures  are  not  possible 
then  abort  the  observation. 

‘o  17.  When  two  mechanics  are  observed  per  task,  one  is  assigned  as  the 

Subject  and  referred  to  as  such  in  the  narrative  text.  The  other 
mechanic  is  referred  to  as  the  Assistant  (AS).  The  assignment  should 
be  made  on  the  basis  of  who  seems  to  be  the  major  actor  at  the 
beginning  of  the  task. 

18.  If  one  or  two  people  are  helping  the  mechanics  at  some  point  in  the 
task,  continue  the  observation.  These  mechanics  should  be  referred 
to  as  assistants  in  the  narrative  text.  (AS2,  AS3,  etc.) 

19.  If  three  or  more  people  are  helping  the  mechanic,  stop  the 
observation  until  the  number  of  helpers  is  down  to  one  or  two,  then 
continue  the  observation.  Note  in  the  narrative  that  there  were  too 
many  people  working  on  the  task  to  keep  track  of  the  flow  of 
information  during  that  time  period. 

20.  If  the  information  flow  of  the  mechanic  and  his  assistant(s)  becomes 
unobservable  because  of  too  many  people  working  on  the  task,  and 
there  is  no  indication  that  the  number  of  people  will  irop  to 
acceptable  levels  (2  helpers),  abort  the  observation. 

21.  In  any  situation  where  people  are  helping  the  mechanic,  the  observer 
must  use  his  judgment  as  to  whether  he  can  keep  track  of  all  the 
activity. 

22.  The  task  is  considered  completed  when  the  mechanic  being  observed 
states  that  it  is. 


Definitions  of  Task-Relevant  Activities 

Those  activities  which  are  recorded  in  narrative  style  during  the 
observation.  Task-relevant  activities  include: 

1.  Task-Relevant  Information.  Refers  to  information  that  enables  the 
mechanic  to  complete  part  or  all  of  the  maintenance  task  he  is 
presently  performing.  It  also  refers  to  information  that  permits  the 
mechanic  to  verify  his  past  work  was  correctly  performed. 

2.  Task-Relevant  Behavior.  Refers  to  behavior  of  the  mechanic  that  is 
directed  toward  completion  of  part  or  all  of  the  maintenance  task  he 
is  presently  performing.  These  are  actions  directed  toward  any 
tools,  parts,  equipment,  or  similar  equipment,  or  information  whether 
they  appear  productive  or  not. 
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3.  Task-Relevant  Observations.  Refers  to  all  task-related  observations 
made  by  the  mechanic  as  he  performs  a  maintenance  task. 

4.  Task-Relevant  Conversation.  All  conversation  addressed  to  the 
mechanic  performing  the  task  or  addressed  by  that  mechanic  to  any 
other  person  If  some  part  of  the  subject  matter  of  conversation 
pertains  to  task-relevant  Information,  whether  phrased  as  a  question 
or  not,  excludes  obvious  non-relevant  Information  or  questions. 


Directions  and  Criteria  of  Directions  of  Information  Flow 

1.  Volunteered  Information.  Task-relevant  Information  volunteered  by 
the  mechanic's  asslstant(s) ,  or  by  some  other  person. 

2.  Requested  Information.  Task-relevant  information  actively  sought  or 
requested  by  the  mechanics. 


General  Information  Sources 

1.  External  to  Task.  ISB  is  considered  external  to  the  task  when  the 
mechanic  receives  information  from  an  external  source. 

2.  Internal  to  Task.  ISB  is  considered  internal  to  the  task  when  the 
mechanic  receives  information  from  the  task  process,  i.e.,  from 
observation  or  manipulation  of  the  equipment  itself. 

3.  Discussion.  Refers  to  discussions  in  which  the  mechanic  and  one  or 
more  persons  talk  over  the  task  but  it  is  not  clear  who  is  requesting 
or  giving  information  (i.e.,  information  flow  directions  are 
confused) . 


Specific  Information  Sources 

1.  Books  (external).  Refers  to  any  consulting  with  technical  manuals 
where  there  Is  written,  photographic  or  graphic  material  presented. 

2.  Person  (external).  Refers  to  any  interaction  with  other  personnel 
during  which  task-relevant  information  is  obtained.  The  information 
includes  that  volunteered  by  a  supervisor  as  well  as  information 
sought  by  the  mechanic  so  long  as  it  is  relevant  to  the  task  the 
mechanic  is  performing. 

3.  Equipment  Data  (external).  Refers  to  events  in  which  an  equipment 
data  plate  is  consulted. 

4.  Equipment  Model  (external).  Refers  to  events  in  which  a  part/area 
of  another  or  the  same  vehicle,  identical  or  similar  to  the 
part/area  the  mechanic  is  presently  working  on,  is  consulted  (e.g. 


looking  at  the  left  wheel  of  a  truck  to  see  how  the  rubber  boot  on 
the  right  wheel  should  be  installed). 

5.  Job  Aids  (external).  Refers  to  any  consulting  with  portable  charts, 
diagrams  not  in  manual,  or  data  cards  that  are  task-specific  (e.g., 
data  cards  that  list  torque  values). 

6.  Process  Provided  (internal).  Refers  to  events  in  which  the  mechanic 
"checks"  or  "inspects"  parts/area  of  the  equipment  being  worked  on. 
These  events  are  made  visible  by  overt  behavior  of  the  mechanic  as 
he  performs  the  required  tasks  on  the  equipment  (e.g.,  rotates  a 
cover  plate  to  Identify  the  correct  alignment  of  bolt  holes). 


Criteria  for  Overt  Non-Verbal  Behavior  to  be  Scored  as  Internal  ISB 

1.  Manual  Manipulation  of  Part.  Characterized  by  rotating,  turning, 
inverting,  shaking,  spinning  or  squeezing  the  parts  and  then 
attempting  to  align  or  fit  them  into  position  on  the  equipment 
(e.g.,  aligning,  or  fitting  throttle  linkage  until  it  falls  into  the 
proper  holes). 

2.  Mechanical  Manipulation  of  Equipment.  Characterized  by  engaging  a 
mechanical  device  on  the  equipment,  which  in  turn  activates  another 
mechanism  (e.g.,  engaging  the  starter  to  turn  over  the  engine),  so 
the  action  of  the  mechanism  can  be  observed. 

* 

3.  Test  Equipment  Readings.  Characterized  by  using  a  mechanical  or 
electrical  device  to  measure  or  record  task-related  information  from 
the  equipment. 

4.  Directed  Observation.  Characterized  by  the  mechanic  holding  a  part 
steadily  and  concentrating  his  attention  on  some  point  on  the  part 
(e.g.,  holding  a  lever  steadily  so  he  sees  the  critical  markings  on 
it);  or  by  the  mechanic  stopping  his  movement  around  the  equipment 
and  concentrating  his  attention  on  some  area  of  the  equipment  (e.g., 
stopping  to  listen  for  a  clanking  sound  coming  from  the  rear  end). 

5.  Non-dlrected  Observation.  Characterized  by  moving  around  the 
equipment  searching  for  task-relevant  information  (e.g.,  walking 
around  a  jeep  sniffing  for  burning  electrical  wires). 


Definitions  and  Examples  for  Types  of  Information  Sought  or  Volunteered 

1.  Technique  for  a  Task  Step.  Questions  concerning  how  to  complete  the 
task  step  presently  being  performed.  Examples  include  information 
about  how  to  remove  a  brake  shoe  clip,  how  to  detach  a  universal 
joint,  or  special  precautions  to  be  followed. 


2.  Task  Steps  Required  for  Completion.  Questions  about  what  the  next 
step  in  the  task  is.  Step-by-step  information  seeking  is 
distinguished  from  how-to  seeking  by  whether  the  question  addresses 
"what  to  do  next?”  That  is,  if  the  mechanic  seeks  information  about 
how  to  successfully  complete  the  action  presently  engaged  in,  it  is  a 
Technique  event.  If  the  mechanic  completes  a  step,  then  seeks 
information  about  what  to  do  next,  it  is  a  Task  Steps  Required  for 
Completion  event. 

3.  Location/Identification  of  Components.  Questions  about  the 
nomenclature  of  task-related  hardware  items  and  where  they  are 
located  on  the  equipment  being  worked  on. 

4.  Formal  Specification  Data.  Questions  about  the  range  of  conditions 
and  indications  for  a  device  operating  within  acceptable  limits. 
Examples  of  specification  information  include  torque  values, 
electrical  values,  and  pictures  of  acceptable  and  unacceptable  spark 
plug  conditions. 

5.  Data  Flow.  Questions  about  the  functional  relationship  between 
components  and  equipment  items,  and  how  a  given  operational  device 
acts  upon  a  given  input  to  produce  a  desired  output.  Examples  would 
include  wiring  diagrams,  schematic  diagrams,  information  showing  the 
components  involved  in  a  given  functional  unit,  and  information 
explaining  the  theory  of  operation  of  equipment  (e.g.,  how  it 
works) . 

6.  Internal/ From  Task.  Information  seeking  when  the  exact  type  of 
information  being  sought  cannot  be  discerned.  This  information  type 
is  most  often  sought  from  a  Task  Process  Source. 

7.  Help  on  Serviceability  Judgment.  Questions  about  whether  or  not  an 
equipment  part  or  assembly  is  serviceable  in  its  present  condition. 
Examples  include  such  questions  as,  "Are  these  bearings  OK?",  or  "Can 
I  use  this  gasket  again?" 

8.  Help  on  Alignment  Judgment.  Questions  about  whether  or  not  an 
adjustment  has  been  completed  correctly  or  whether  equipment  parts  or 
assemblies  are  correctly  positioned  (aligned).  Examples  include  such 
questions  as,  "Is  this  the  right  brake  pressure?",  or  "Is  this  road 
wheel  on  all  the  way?" 


Outside  Support  Information  Sought  or  Volunteered.  When  Technique,  Task 
Steps  Required  or  Specifications  information  is  sought,  questions 
concerning  any  special  implemerts,  instruments,  provisions,  or  other 
necessities  required  to  complete  a  given  task  -  or  needed  to  facJ'itate 
performance  of  a  given  task  should  be  indicated  with  the  outside  support 
code,  in  addition  to  the  appropriate  information  type  code.  Examples 
include  information  about  using  a  torque  wrench  or  meter,  how  many  helpers 
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are  necessary,  and  which  lubricants  or  solvents  are  needed.  This 
definitely  Includes  how  to  use  or  read  test  equipment. 


Definitions  and  Examples  for  Sources  of  Errors 


OUTSIDE  -  Refers  to  errors  that  arise  from  a  previous  error  by  another 
person  or  when  another  person  gives  incorrect  information 
which  is  used  by  the  mechanic.  An  example  is  stripping  a 
bolt  that  has  been  cross-threaded  during  prior  installation, 
or  following  incorrect  torque  instructions. 

EXPEDIENT  -  Refers  to  an  error  that  arises  from  tool,  part,  or 

information  unavailability.  Examples  include  not  torquing 
bolts  to  specifications  because  no  one  has  a  torque  wrench, 
or  applying  oil  to  threads  of  access  plugs  before 
reinstalling  because  antiseize  compound  is  not  available. 


MECHANIC  -  Refers  to  those  errors  which  arise  from  the  individual 

mechanic's  own  lack  of  knowledge  or  carelessness.  Examples 
include  pulling  hoses  without  draining  fluid  reservoir  first, 
thus  spilling  antifreeze  all  over  floor;  removing  parts 
without  disconnecting  electrical  connections,  thus  often 
breaking  wires;  and  not  bothering  to  find  out  if  bolts  should 
be  tightened  to  a  specific  torque  value. 


Definitions  and  Examples  for  Process-Error  Types 

1.  Violate  Good  Mechanical  Practice.  Errors  made  when  the  mechanic 
violates  good  general  mechanical  practice.  These  are  errors  which 
often  lead  to  damaged  parts  or  sloppy  workmanship.  Examples  include 
improperly  greasing  wheel  bearings,  and  failing  to  drain  oil 
reservoir  before  attempting  to  change  primary  oil  filter. 

2.  Wrong  Technique  Used.  Errors  when  the  mechanic  uses  the  wrong  tools 
or  procedure  during  the  task  process.  Errors  of  this  type  often  lead 
to  damaged  equipment  parts.  Examples  include:  not  using  a  sling  to 
support  heavy  equipment  parts  being  removed  or  installed,  and  prying 
with  a  screwdriver  to  remove  an  oil  filter  element — damaging  the 
element. 

3.  Specification  Error.  Process  errors  made  when  the  mechanic  does  not 
follow  exact  specification  requirements  stated  in  the  Task  Manual. 
Examples  include  adjusting  contact  breaker  points  to  an  incorrect  gap 
width,  or  tightening  cylinder  head  bolts  to  an  incorrect  or  unknown 
torque  value. 

4.  Wrong  Part  or  Component.  Errors  made  when  an  incorrect  equipment 
part  is  installed,  or  an  attempt  is  made  to  install  It.  Occasions 


when  an  equipment  part  is  left  out  of  an  assembly  are  also  Parts 
Errors.  Examples  include  installing  a  secondary  oil  filter  in  the 
primary  filter  case,  or  leaving  out  part  of  the  U- joint  assembly. 


5.  Wrong  Position  or  Orientation  of  Part/Component.  This  error  usually 
occurs  when  equipment  parts  are  installed  in  such  a  place  or  rotated 
so  that  they  cannot  be  properly  seated  and  attached.  An  example  is 
seating  an  oil  cooler  in  such  a  position  that  the  inflow/outflow 
links  cannot  be  attached. 

6.  Wrong  Order-of-Steps .  Errors  made  when  the  mechanic  does  task  steps 
out  of  their  prescribed  order.  (Occasions  when  task  steps  are  com¬ 
pletely  omitted  are  also  Order-of-Steps  errors.)  These  errors  often 
resemble  Recursive  Errors  until  the  mechanic  realizes,  or  is  told,  of 
the  missed  step.  Examples  include  repeated  efforts  to  pull  off  a 
brake  drum  before  contracting  brake  shoes,  or  repeated  attempts  to 
pull  out  generator  before  removing  all  attached  wires. 

7.  Wrong  Adjustment  Technique.  Errors  made  when  the  mechanic  uses  a 
wrong  tool  or  procedure  to  complete  adjustment  of  an  equipment  part. 
Examples  include  turning  an  adjustment  the  wrong  way  to  tighten/ 
loosen  it,  and  not  jacking  up  a  jeep  before  adjusting  the  wheel 
bearings. 


Definitions  of  Modifiers  of  Process  Error  Types 

Any  action  coded  as  one  of  the  above  error  types  might  also  be  recursive 
and/or  destructive. 

1.  Recursive  Error.  A  recursive  error  is  defined  by  a  mechanic  making 
one  of  the  above  error  types  two  or  more  times  while  working  on  the 
same  component  part,  such  as  when  working  on  the  bearings  of  one 
wheel  assembly.  This  should  be  distinguished  from  occasions  when  the 
same  mistake  is  made  once  on  each  of  two  or  more  "mirror  image” 
component  parts  on  the  same  vehicle,  such  as  the  same  mistake  on  each 
of  the  four  wheel  assemblies.  In  this  second  situation  the  errors 
made  during  work  on  each  mirror  image  part  should  be  coded  as 
individual  errors. 

2.  Destructive  Error.  An  error  is  destructive  when  either  a  mechanic 
does  something  that  immediately  results  in  damage  to  an  equipment 
part,  or  does  something  which  will,  over  time,  decrease  the 
servicable  life  of  the  equipment  part  and  vehicle.  Examples  of  the 
first  include  tearing  a  gasket  or  rounding  off  a  bolt  head  or  nut. 

Not  packing  dry  bearings  or  using  the  wrong  filter  element  are 
examples  of  the  second. 


1.  Serious  Uncorrected  Errors.  Any  error  which  was  left  uncorrected 
when  the  mechanic  stated  that  the  job  was  finished  and  which,  in  the 
opinion  of  expert  mechanics,  would:  (a)  shorten  the  serviceable 
vehicle  life,  (b)  immediately  endanger  the  vehicle,  (c)  endanger 
drivers  or  passengers  in  the  vehicle,  or  (d)  endanger  by-standers 
near  the  vehicle. 

2.  Checkout  Possible  -  Not  Made.  This  is  when  the  mechanic  does  not 
attempt  to  complete  a  checking  of  the  equipment  part  that  has  been 
installed  or  adjusted.  What  tasks  should  have  a  checkout  depends  on 
the  task  itself,  the  operational  status  of  the  vehicle  apart  from  the 
task  at  hand,  and  pertinent  regulations.  Examples  include  not 
testing  the  brakes,  and  not  starting  the  vehicle  and  checking  for 
leaks  in  the  coolant  system. 


Data  Transcription  Procedure 


1.  The  purpose  of  the  data  transcription  is  to  transform  the  raw 
narrative  data  into  an  easy-to-read,  concise,  step-by-step  format. 

You  will  use  a  pen,  so  that  the  transcripts  can  be 

copied  later. 

2.  Record  observation  number,  MOS/duty  position,  time,  date,  task  name, 
number  of  times  task  performed,  time  since  task  last  performed,  and 
time  in  MOS  in  the  spaces  provided  at  the  top  of  the  data 
transcription  sheet. 

3.  Be  sure  to  record  the  observation  number  and  page  number  at  the  top 
of  all  subsequent  pages. 

4 .  Detail  each  step  sufficiently  so  that  the  actual  series  of  events  can 
be  easily  understood.  This  is  the  most  important  function  of  the 
transcriptions . 

5.  Number  each  step. 

6.  When  an  information-seeking  event  occurs,  code  the  step  with  the 
appropriate  code.  When  the  information  which  was  sought  was  not 
obtained,  the  narrative  should  make  clear  whether  the  information  was 
not  available,  or  whether  obtained  information  was  not  useful. 

7.  When  a  step  includes  the  use  of  a  TM,  record  the  TM  number,  page 
number,  paragraph  number,  figure  number,  and  date  in  the  data 
transcription. 


8.  When  a  step  requires  a  non-information  seeking  behavior  code,  code 
the  step  in  the  same  manner  as  for  the  observation  narrative  data. 


EXAMPLE 

OBSERVATION  DATA  SHEET 


0  #:  _ 

MOS/DUTY  POS.J  _ 

NO.  TIMES  TASK  PERFORMED: 

TIME  SINCE  LAST  TASK:  _ 

TASK: 


INITIALS 

DATE:  _ 

TIME:  _ 

TIME  IN  MOS 


Steps  Code 

Task  Decision  Information:  (Symptom?  Selection  Criteria?  Other 
possible  actions?) 


Task  Performance: 


9.  When  coding  an  error,  determine  the  step  in  the  task  where  the  error 
was  made  and  place  the  code  at  that  step. 

10.  If  the  process  error  is  also  Destructive  and/or  Recursive  error,  the 
appropriate  code  should  also  be  marked.  Special  attention  should  be 
given  to  Recursive  Errors  as  noted  previously  in  Definitions  of 
Modifiers  of  Process  Errors. 

11.  If  the  error  is  also  a  Serious  Uncorrected  Error,  the  appropriate 
code  should  be  placed  in  the  code  column. 


DATA  TRANSCRIPTION  CODES 


Information  Codes:  (Place  in  code  blank  with  slashes  between  symbols 

and  in  the  following  order,  e.g.,  R/E/PB/H: 


V/R 

V  «■  Volunteered  Information 
R  *  Requested  Information 

I/E/D 

I  =*  Internal  to  Task 
E  ■  External  to  Task 
D  =  Discussion  about  Task 

SOURCE 

PB  =  Person,  Supervisory  (Boss) 

PF  »  Person,  Fellow  Enlisted  Man 
D  *  Data  Plate 

E  -  Equipment  Model  (same  or  different  piece  of  equipment) 
J  =  Job  Aid 

F  =  From  Task  Process  (not  otherwise  identifiable) 

TM  =*  Tech.  Manual  (Get  TM  if,  page  if,  paragraph  or  figure  #) 


TYPE 

T  *  Technique  for  a  Task  Step 
P  =  Procedural/ Step-by-Step 
G  *  Geography  of  Equipraent/Parts  I.D. 
S  =  Specification  Data 
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D  -  Data  Flow 

I  *  Internal  from  Task  (not  otherwise  Identifiable) 
SJ*  Make  the  judgment  for  me  on  serviceability 
AJ*  Make  the  judgment  for  me  on  ad justment/allgnment 

OUTSIDE  SUPPORT  INFORMATION 

0  *  Outside  Support  Information 


Error  Codes;  (Place  code  In  blank  with  slashes  between  symbols  and 
In  the  following  order,  e.g.,  MC/ A/DE/AC.) 


Source'  of  Errors 


OT  *  Previous  Errors  by  Others  or  Incorrect  Information 
EX  -  Expedient  Error 
MC  ■  Mechanic's  Error 


Process  Errors 


G  -  Violate  Good  Mechanical  Practice 
T  =  Wrong  Technique  Used 
S  *  Specification  Error 
P  -  Wrong  Part/ Component 
R  -  Position/Rotation  Error 

0  *  Wrong  Position  or  Orientation  of  Part/ Component 

A  «■  Wrong  Adjustment  Technique 


Modifiers  of  Process  Error  Types 
RE  -  Recursive  Error 
DE  ■  Destructive  Error 

Completion  Errors 


SUE  *  Serious  Uncorrected  Error 
CN  *  Checkout  Possible/Not  Made 

Non-Information  Seeking  Behavior  Codes 

J1  *  Serviceability  Judgments 
J2  ■  Ad just/ Align  Judgments 
AS  n  ■  Assigned  by  ji  persons  during  this  step 

X  ■  Information  sought  is  not  what  was  obtained 


EXAMPLE 


DATA  TRANSCRIPTION 

SHEET 

Initials 

0  #: 

MOS/DUTY 

TIME 

DATE: 

TASK 

CLOZE: 

NO.  TIMES  TASK 

TIME 

SINCE  LAST 

CIV.  EXP 


TIME  IN  MOS 


MM 


GT 


INSTRUCTIONS  FOR  OBSERVING  TEAMS 


A  team  Is  defined  as  three  or  more  mechanics  assigned  to  work,  on  the 
same  automotive  task.  This  situation  is  different  from  one  in  which  there 
is  one  principal  mechanic  with  one  or  more  assistants  helping  at  various 
times. 


The  procedures  outlined  in  Instructions  for  Observers  are  applicable 
to  a  team  situation,  with  the  following  exceptions. 


Observation  Procedure  for  Teams 

1.  Two  observers  are  necessary  to  keep  track  of  the  work  flow,  due  to  the 
number  of  mechanics  at  work  and  because  the  mechanics  may  be  engaged 
at  different  locations  around  the  vehicle  or  work  area. 

2.  Observers  should  agree  on  labels  for  the  mechanics,  e.g..  Mechanic  A, 
Mechanic  B,  etc. 


General  Information  Sources  for  Teams 

Internal  to  Task  (I).  ISB  is  considered  internal  to  the  task  when  the 
mechanic  receives  information  from  the  task  process. 

External  to  Task/Within  a  Team  (TE) .  ISB  is  considered  external  to  the 
task  and  from  within  a  team  when  the  mechanic  receives  information  from 
another  member  of  the  team. 

External  to  Task  (E).  ISB  is  considered  external  to  the  task  when  the 
mechanic  receives  information  from  some  source  outside  the  team. 

Discussion  Within  a  Team  (TD).  Refers  to  discussions  in  which  two  or  more 
team  members  talk  over  the  task,  but  it  is  not  clear  who  is  requesting  or 
giving  information  (i.e.,  information  flow  directions  are  confused). 

Discussion  (D).  Refers  to  discussions  involving  one  or  more  team  members 
and  one  or  more  persons  who  are  not  members  of  the  team,  but  it  is  not 
clear  who  is  requesting  or  giving  information  (i.e.,  information  flow 
directions  are  confused). 
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Transcription  Procedure  for  Teams 


OBSERVER  TRAINING  PROCEDURE 


Purpose 


The  purpose  of  this  procedure  is  to  train  new  observers  in  this 
unobtrusive  data  collection  technique. 

Technique  Description.  This  is  a  low-reactivity  (unobtrusive)  data 
collection  technique.  It  is  designed  for  the  collection  of  complete 
narrative  descriptions  of  task  performances  at  the  usual  job  site  under 
essentially  normal  work  conditions.  That  is,  every  effort  has  been  made 
to  avoid  the  appearance  of  a  test  or  an  evaluation  of  either  the  worker  or 
his  organization.  The  primary  data  of  interest  are:  the  use  of  technical 
information,  where  the  information  is  sought  and  obtained,  what  kind  of 
information  is  sought  and  obtained,  and  what  kinds  of  errors  occur.  The 
secondary  data  of  interest  are  the  context  in  which  technical  information 
is  used  and  sought.  The  “context**  refers  to  both  where,  in  the  particular 
task,  the  information  is  used  or  sought  and  to  what  the  environmental  and 
work  circumstances  are. 

Training  Goals.  The  primary  goal  of  this  training,  as  of  most 
training,  is  to  reduce  variation  in  the  performance  of  the  data  collection 
from  observer  to  observer.  There  must  be  commonality  in  the  selection  of 
which  behavior  to  record  or  not  record  during  an  observation.  There 
should  be  a  minimum  information  base  which  every  observer  brings  to  the 
observation  with  him.  The  minimum  base  must  include  basic  mechanical 
practice,  familiarity  with  vehicles  to  be  observed,  familiarity  with  the 
tasks  to  be  observed,  familiarity  with  the  Army  Motor  Pool  structure  and 
systems,  familiarity  with  the  definitions  and  codes  for  data  collection, 
and  an  understanding  of  the  task  process  model  being  used. 

Training  Procedure.  The  training  of  a  new  observer  starts  with  an 
overview  of  the  purpose  and  goals  of  data  collection.  Next  the  new 
observer  gets  an  explanation  of  the  methodology,  from  development  to 
analysis.  After  familiarizing  the  observer  with  the  methodology  of  the 
front-end  analysis,  the  observer  is  given  hands-on  experience  with  the 
write-up  of  typical  tasks  from  the  technical  documentation  (as  step-by- 
step  task  descriptions).  He  then  rates  the  tasks  for  information  demand. 
Finally,  the  observer  familiarizes  himself  with  the  existing  body  of  task 
write-ups,  demand  ratings,  and  observations  previously  collected.  The 
last  step  is  practice  observations  for  discussion  and  critique. 
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Example  of  ASA'b 
Training  Schedule 

A.  First  Day 

1.  Overview  of  purpose  and  goals  of  the  project  and  the  data 

collection.  1  hour  -  Project  Director  (PD)  and  staff. 

a.  Purpose:  Provide  advice  and  guidance  to  writers  of  technical 
manuals. 

b.  Goal  1:  Gather  baseline  data  on  use  of  and  need  for 
technical  Information  under  job  performance  conditions. 

c.  Goal  2:  Find  out  why  and  by  whom  tasks  are  usually  assigned, 
as  well  as  completion  cues  and  checkout  process  for  task 
completion. 

d.  Goal  3:  Collect  data  on  Information-seeking  behavior  (ISB)  by 
type  and  source,  errors,  and  other  personnel  variables  of 
Interest. 

e.  Goal  A:  Determine  the  factors  which  allow  prediction  of  ISB 
by  type  and  source. 

2.  Explanation  of  methodology.  1  hour  -  PD  and  staff 

a.  Task  process  model  -  the  conceptual  framework  for  the  method¬ 
ology. 

b.  Evolution  of  narrative  recording  of  unobtrusive  observation. 

c.  Stages  of  front-end  analysis  which  precede  observation. 

d.  Methods  and  goals  of  analysis  of  collected  data  (what  do  we 
do  with  the  data). 

3.  Description  of  the  Army  and  Array  Motor  Pools  as  they  affect  our 

data  collection.  2  hours  -  PD  and  staff 

a.  Officer  and  E-M  rank  and  meaning. 

b.  Structure  of  Army  Infantry  Divisions 

c.  Structure  of  motor  pool 

d.  Military  terminology  and  slang 
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B.  Second  Day 

1.  Discussion  of  task  analysis  procedure.  1  hour  -  staff 

a.  Q&A  clarification  time. 

2.  Introduction  of  technical  manuals  (TM).  2  hours  -  staff 

a*  Comparison  of  a  task  description  write-up  and  the  same  task 
as  described  in  the  TM. 

b.  Discussion  of  how  to  use  TM  for  basis  of  task  description. 

3.  Practice  task  description  write-up.  2  hours  -  staff 

a.  Independent  applications  of  prior  learning  to  write  up 
practice  task. 

b.  Comparison  of  previous  write-ups  and  critique  (feedback). 

4.  Practice  task  description  write-up  for  five  tasks  on  jeep- 

TM9-2320-218-20.  4  hours. 

C.  Third  Day 

1.  Comparison  and  critique  session  on  jeep  tasks.  1  hour  -  PD  and 
staff 

2.  Practice  task  description  write-up  for  M113  Armored  Personnel 
Carrier  and  M60A1  Tank  -  five  tasks  each  -  remainder  of  day. 

D.  Fourth  Day 

1.  Complete  practice  write-ups.  2-4  hours. 

2.  Feedback  session,  reliability  rating  with  previous  analyses. 

1  hour  -  PD  and  staff 

3.  Feedback  session.  1/2  hour  -  staff 

4.  Reliability  rating  with  previous  analyses,  staff 

E.  Fifth  Day 

1.  Practice  making  information  demand  ratings  (IDR)  on  first  task 
write-ups.  1/2  hour 

2.  Feedback  and  discussion  of  IDR.  1/2  hour  -  staff 
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3.  Practice  IDR  on  other  14  tasks.  6  hours  maximum 

4.  General  Q&A  and  discussion  of  front-end  analysis  procedure. 

1  hour  -  PD  and  staff. 

5.  Study  "Instructions  for  Observers"  booklet  -  ad  lib 

F.  Sixth  Day 

1.  Discussion  of  observation  technique  and  procedures.  2  hours 
PD  and  staff 

2.  Study  existing  body  of  observations  and  transcriptions.  6  hours 
assisted  by  staff 

G.  Seventh  Day  -  Ninth  Day 

1.  Study  existing  body  of  task  descriptions  and  observations  - 
study  relations  between  the  two  -  all  day  ad  lib. 

2.  Discussion  of  lore  of  actual  data  collection.  As  needed  -  staff 


H.  Tenth  Day 

1.  Discussion.  3  hours  -  PD  and  staff 


I.  First  Day  of  Field  Data  Collection 

1.  Practice  observation  and  reliability  tests  with  fellow  team 

members .  8  hours 

2.  Discussion  and  consensus  correction  of  narrative  and  transcrip¬ 
tion.  2  hours. 
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